load_aware_locality: implement lb policy#45725
Conversation
|
/coverage |
|
Coverage for this Pull Request will be rendered here: https://storage.googleapis.com/envoy-cncf-pr/45725/coverage/index.html For comparison, current coverage on https://storage.googleapis.com/envoy-cncf-postsubmit/main/coverage/index.html The coverage results are (re-)rendered each time the CI |
ac7a860 to
2f0a702
Compare
6221cab to
8acb258
Compare
|
/coverage |
|
Coverage for this Pull Request will be rendered here: https://storage.googleapis.com/envoy-cncf-pr/45725/coverage/index.html For comparison, current coverage on https://storage.googleapis.com/envoy-cncf-postsubmit/main/coverage/index.html The coverage results are (re-)rendered each time the CI |
jukie
changed the title
|
/gemini review |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request implements the envoy.load_balancing_policies.load_aware_locality load balancer policy, which weights localities based on ORCA-derived utilization headroom and supports in-band ORCA reporting across all priority levels. The changes include configuration parsing, main-thread and worker-local load balancing logic, and comprehensive unit, configuration, and integration tests. Feedback on the pull request points out that the integration test target in the BUILD file should list the envoy.load_balancing_policies.round_robin extension in its extension_names attribute, as it is instantiated during the integration test.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
fb02adb to
68310e2
Compare
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request implements the load_aware_locality load balancer policy, which weights localities by ORCA-derived utilization headroom, consumes in-band ORCA reporting, and applies at all priority levels. The implementation includes the main-thread LoadAwareLocalityLoadBalancer for computing locality routing weights and publishing snapshots, and the worker-thread WorkerLocalLb for delegating endpoint selection to child load balancers. Comprehensive unit, configuration, and integration tests have been added to verify the functionality. There are no review comments to address, so no additional feedback is provided.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Introduce the load-aware locality LB extension: typed config, factory registration, per-endpoint metadata, and the LoadAwareLocalityLbConfig that carries tuning knobs to the balancer. Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>
Add LocalityLbHostData: workers store per-host utilization from ORCA load reports with release/acquire publication for the main thread. Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>
Add the per-priority routing-weights snapshot types and the worker LB factory that publishes snapshots to workers via thread-local storage. Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>
Add WorkerLocalLb: live priority/health/panic selection on workers, then index-aligned locality choice delegating to a per-locality child LB. Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>
Add the main-thread balancer that periodically computes EWMA-smoothed locality weights from host utilization and publishes routing snapshots. Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>
Unit coverage for config parsing and the balancer's weight computation, locality selection, and host-sync paths. Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>
End-to-end integration coverage for the load-aware locality policy. Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>
Document the policy's design and add the changelog entry. Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>
68310e2 to
9184e60
Compare
| - **Out-of-band ORCA reporting is not yet supported.** Only in-band | ||
| (per-response) ORCA reports are consumed. ``enable_oob_load_report`` and | ||
| ``oob_reporting_period`` are accepted but currently have no effect. | ||
| - **Probing is required.** A locality only produces fresh ORCA samples | ||
| when it receives traffic, so ``remote_probe_fraction`` must stay above 0 | ||
| to keep remote localities reporting. Set it to 0 only when cross-zone |
There was a problem hiding this comment.
Depends on #45953 as noted in PR description. Once that's merged I'll wire OOB into this policy and update the docs again.
Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>
|
/retest |
2 participants
Commit Message: load_aware_locality: implement ORCA utilization-based locality picking
Additional Description:
Implements the envoy.load_balancing_policies.load_aware_locality extension added in #44341 which is a locality-picking LB policy that weights localities by backend utilization aggregated from ORCA load reports.
Some notes:
Risk Level: low/medium - opt-in extension that is still work-in-progress
Testing: Added unit and integration tests, also verified in a real environment.
Docs Changes: Un-orphaned docs/root/intro/arch_overview/upstream/load_balancing/load_aware_locality.rst which will now be rendered in site docs
Release Notes: Added
Platform Specific Features: N/A
Fixes #43665
AI disclosure: portions of this PR and it’s tests were developed with LLM but I have reviewed and understand the code